…aNet (#2484)

* fix(moe): preserve fp32 A_log in Qwen3.5-MoE and Qwen3-Next GatedDeltaNet

GatedDeltaNet A_log/dt_bias are intrinsically fp32, but the native MoE
Qwen3.5-MoE and Qwen3-Next models sharded through moe/parallelizer.py with a
single bf16 mixed-precision policy, silently rounding these params to bf16 and
degrading the exponentiated decay rate.

Generalize the dense Qwen3.5 _fp32_params holder mechanism into a shared helper
(components/models/common/gated_delta_net_fp32.py), isolate the fp32 GDN params
into per-module holders at parallelize time, and shard each holder as a
dedicated fp32 FSDP unit in apply_fsdp (preserving EP ignored_params). Add an
fp32-aware Qwen3NextGatedDeltaNet subclass that routes the gate through the
holder, and strip the _fp32_params prefix on save so checkpoints stay HF-clean.

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(moe): avoid pre-reading holder-owned dt_bias in Qwen3.5-MoE gate

CPAwareGatedDeltaNet._compute_gate evaluated self.dt_bias before entering the
holder forward. Once isolate_fp32_params moves an intrinsically-fp32 dt_bias into
the _fp32_params holder, the patched __getattr__ makes self.dt_bias resolve to
the holder-owned param, reading it outside the holder forward (before FSDP
unshards), which defeats the fp32 isolation. The holder forward already prefers
its own dt_bias, so route through holder(a) when it owns dt_bias and only pass
self.dt_bias when dt_bias stays a bare (bf16) param -- mirroring Qwen3-Next.

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* test(moe): cover apply_fsdp fp32 GDN branch; exclude GPU-only Next forward

Add a CPU unit test that drives the apply_fsdp fp32 branch (isolation reported,
fp32 holder policy built, per-block holder sharded) to lift Codecov patch
coverage reliably on every PR. Mark Qwen3NextFp32GatedDeltaNet.forward with
``# pragma: no cover`` since it is a verbatim HF GDN forward that requires CUDA
conv1d/FLA kernels and is only exercised by GPU/functional runs.

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(qwen): preserve GDN fp32 params under FSDP2

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(qwen): scope GDN fp32 isolation to linear attention

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(qwen): mark fp32 GDN modules explicitly

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* docs(model): clarify GDN fp32 marker contract

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(nemotron): keep router correction bias fp32

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(dtype): preserve fp32 values during model casts

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* refactor(models): make qwen-next own fp32 gdn holder

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* docs(skills): clarify fp32 holder ownership

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(checkpoint): tolerate missing TE extra state

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(models): preserve duplicate fp32 buffers

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

---------

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>
(cherry picked from commit 174c5d1)
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>